Statistical and Algorithmic Approaches to Bullet Matching

Problem Statement

Comparison Microscope

Same Source Problem: were two bullets fired through the same gun barrel?
Out approach similar to Bachrach et al. (2010): automatic matching introduces objective approach to matchign
Automatic matching based on 3d topographic images
Score in [0,1] with empirical distribution of known matches and known non-matches

x3p format: ISO 25178-72:2017

Automated Matching Algorithm

Matching is based on features extracted from aligned land-to-land comparison: ccf (Vorburger et al. 2011), total variation (Clarkson and Adams 1933), number striae, cms (Biasotti 1959), number non-alignments, …
Used Random Forest (Breiman 2001), but other machine learning algorithms show similar performance; features are much more important than the algorithm

Reference database

James Hamby Study: Ten consecutively rifled Ruger P-85 pistol barrels used to fire 20 “known” test bullets and 15 “unknown” bullets for comparison (Hamby, Brundage, and Thorpe 2009)
Use 3d image of Hamby-252 as reference database to train the algorithm
Extract features for all land-land comparisons (21945 total)
Compare differences in features between 7,075 known matches and 14,040 known non-matches

Results: Distributions of Known Matches and Known Non-Matches

Algorithm: Forest of 1000 trees

Algorithm is based on 1000 trees, each tree gets a vote

Automated Matching Algorithm: Front-End Web Application

https://isu-csafe.stat.iastate.edu/shiny/bulletr/

Testing the Model

Phoenix PD: 8 barrels (with 2 test firings), 2 unknowns
3d imaging by Sensofar (thanks, Bill!)
Scans by different operator, on different microscope, different staging of the bullets, different resolution
Results are promising

Testing the Model (cont’d)

How much land do we need for a match?

error rates seem stable down to about 50% of a land
middle of the land seems slightly better (not significant)

Limitations

biggest limitation thus far is limited number of available 3D scan data for bullets:

in NIST ballistics database (Xiaoyu Alan Zhang, https://tsapps.nist.gov/NRBTD):

Hamby Set 252 (10 Barrels)
Hamby Set 44 (Same 10 Barrels)
Cary Persistence (1 Barrel)

11 unique gun barrels is not yet enough to form a true reference distribution for known matches and non-matches…

…However, the structure of the database means that as soon as new data is available, the features and scores can be easily recomputed.

please contribute your experimental data!

Transparency & Reproducibility

Open Data: NIST ballistics database, locally we are in the process of (re-) scanning Hamby-44 and Hamby-259
Open Algorithms: bulletr is R package on CRAN with functionality for read/write x3p format, feature extraction and matching algorithms, devel version on github
Open Results: local database with processed land scans; actual future work: submit scores for land-to-land comparisons from different approaches/institutions/labs to database

Future work

improve matching:
- importance of parameter choices
- expand on features, e.g. adapt toolmark scores to bullet lands
- profile extraction (groove detection, area of extraction)
investigate sources of (statistical) error: how much variability is introduced due to operator, lab, type of microscope? (ties into efforts by Martin Baiker from NFI and NIST)
expand on applications: e.g. primer shearing marks?

Thank You

Special thanks to Alan Zheng at the National Institute of Standards and Technology for maintaining the NIST Ballistics Toolmark Research Database and providing many useful suggestions.

References

Bachrach, Benjamin, Anurag Jain, Sung Jung, and Robert D Koons. 2010. “A Statistical Validation of the Individuality and Repeatability of Striated Tool Marks: Screwdrivers and Tongue and Groove Pliers.” Journal of Forensic Sciences 55 (2): 348–57.

Biasotti, Alfred A. 1959. “A Statistical Study of the Individual Characteristics of Fired Bullets.” Journal of Forensic Sciences 4 (1): 34–50.

Breiman, Leo. 2001. “Random Forests.” Machine Learning 45 (1): 5–32. doi:10.1023/A:1010933404324.

Clarkson, James A, and C Raymond Adams. 1933. “On Definitions of Bounded Variation for Functions of Two Variables.” Transactions of the American Mathematical Society 35 (4). JSTOR: 824–54.

Hamby, James E., David J. Brundage, and James W. Thorpe. 2009. “The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries.” AFTE Journal 41 (2): 99–110.

Vorburger, T.V., J.-F. Song, W. Chu, L. Ma, S.H. Bui, A. Zheng, and T.B. Renegar. 2011. “Applications of Cross-Correlation Functions.” Wear 271 (3–4): 529–33. doi:http://dx.doi.org/10.1016/j.wear.2010.03.030.